Skip to content

WIP: Populate tox with a script #3906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 31 commits into from
Closed

WIP: Populate tox with a script #3906

wants to merge 31 commits into from

Conversation

@sentrivana
Copy link
Contributor

@sentrivana sentrivana commented Jan 8, 2025 •
edited
Loading

WIP

  • Fetch releases for packages to test from PYPI and pick a handful to test against
  • Remove -latest tests; instead, we can rerun the script to regenerate tox.ini as it will now always also include the latest release
    • TODO: actually make an action for this to happen on PRs
  • Add a CI check for making sure the script/template is in sync with the contents of tox.ini (so that folks get notified if they edit tox.ini directly instead of the template/script)
  • ❗Implement an automatic deprecation policy: only test releases that are at most X years old

@codecov
Copy link

codecov bot commented Jan 8, 2025 •
edited
Loading

❌ 284 Tests Failed:

Tests completed Failed Passed Skipped
20845 284 20561 6399
View the top 3 failed tests by shortest run time 
 tests.integrations.cohere.test_cohere
Stack Traces | 0s run time 
ImportError while importing test module '.../integrations/cohere/test_cohere.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../site-packages/_pytest/python.py:599: in _importtestmodule
    mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../site-packages/_pytest/pathlib.py:533: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.6.15.../x64/lib/python3.6/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:994: in _gcd_import
    ???
<frozen importlib._bootstrap>:971: in _find_and_load
    ???
<frozen importlib._bootstrap>:955: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:665: in _load_unlocked
    ???
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../_pytest/assertion/rewrite.py:171: in exec_module
    exec(co, module.__dict__)
.../integrations/cohere/test_cohere.py:5: in <module>
    from cohere import Client, ChatMessage
E   ImportError: cannot import name 'Client'
 tests.integrations.django.test_basic
Stack Traces | 0s run time 
ImportError while importing test module '.../integrations/django/test_basic.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.10-django-v5.1.4/lib/python3.10.../site-packages/_pytest/python.py:493: in importtestmodule
    mod = import_path(
.tox/py3.10-django-v5.1.4/lib/python3.10.../site-packages/_pytest/pathlib.py:587: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.10.16.../x64/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
.tox/py3.10-django-v5.1.4/lib/python3.10.../_pytest/assertion/rewrite.py:184: in exec_module
    exec(co, module.__dict__)
.../integrations/django/test_basic.py:40: in <module>
    from tests.integrations.django.utils import pytest_mark_django_db_decorator
.../integrations/django/utils.py:4: in <module>
    import pytest_django
E   ModuleNotFoundError: No module named 'pytest_django'
 tests.integrations.django.test_cache_module
Stack Traces | 0s run time 
ImportError while importing test module '.../integrations/django/test_cache_module.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.10-django-v5.1.4/lib/python3.10.../site-packages/_pytest/python.py:493: in importtestmodule
    mod = import_path(
.tox/py3.10-django-v5.1.4/lib/python3.10.../site-packages/_pytest/pathlib.py:587: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.10.16.../x64/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
.tox/py3.10-django-v5.1.4/lib/python3.10.../_pytest/assertion/rewrite.py:184: in exec_module
    exec(co, module.__dict__)
.../integrations/django/test_cache_module.py:18: in <module>
    from tests.integrations.django.utils import pytest_mark_django_db_decorator
.../integrations/django/utils.py:4: in <module>
    import pytest_django
E   ModuleNotFoundError: No module named 'pytest_django'

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

@sentrivana
Copy link
Contributor Author

sentrivana commented Jan 13, 2025

Closing, continuing in #3920

@sentrivana sentrivana closed this Jan 13, 2025
@sentrivana sentrivana deleted the ivana/populate-tox-2 branch February 14, 2025 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sentrivana